A Lightweight Supplier Evaluation based on CMMI

Software for international companies is often developed by a multiplicity of suppliers using different approaches and processes. Only a few of these suppliers are already rated according to CMMI, ISO 15504 or ISO 9001. Therefore Vodafone Global and sd&m have developed an approach to evaluate and to monitor the quality of suppliers’ development processes with selfassessments based on a detailed questionnaire. This article gives a short introduction to CMMI, describes the selection of the relevant process areas, the creation of the questionnaire, and the automated evaluation of the results. 1 Motivation, Scope and CMMI Vodafone, the leading global mobile telecommunications company, has many national Operational Companies (OpCo's). The software systems of Vodafone’s Intranet are developed by both internal and external development centers. The underlying development processes are different, the quality and results vary. Vodafone wants to ensure that the different development processes for software projects worldwide will deliver predictable results with high quality. An alignment of the different development processes to a common standard was not possible. Because of this, Vodafone and sd&m have developed an approach to evaluate and to monitor the quality of the development processes. The assessment method used is based on CMMI® (Capability Maturity Model Integration) V1.1. CMMI is a framework of management, engineering, and support best practices organized into 25 topics, called process areas [Cm02]. It has been developed by the Software Engineering Institute (SEI) at Carnegie Mellon University and can be thought of as a set of “process requirements” that help guide an organization to define and implement processes related to the 25 topics. CMMI was chosen as the basis for the assessment due to the fact that it covers various disciplines, it is based on practical best-practices, and it not only defines process requirements but also offers possibilities for constructive assessment of organizations. An international standard appropriate for large organizations was also one of the requirements for choosing the basic framework. Furthermore, there was a trend observed that more and more suppliers start with CMMI improvement programs. So it was convenient to speak the same language. The requirements for assessing the capability according to CMMI are defined in [Cm00]. Since conducting a class A appraisal like SCAMPI [Me01] is very time-consuming, research is concentrating on developing class B or C appraisals [Ha03]. The goal of our work was to define a customized supplier evaluation using CMMI as a basis, not to invent a new general assessment method. To define the terms used in the following sections, we give a short summary of the CMMI concepts. For more details please refer to [CKS02] or [Kn02]. CMMI models are organized in two representations, continuous and staged. The continuous representation measures capability levels whereas in the staged representation an organization has to fulfill some areas to reach a maturity level. In our work we have chosen the continuous representation, since in this, an organization can select the process areas in which it wants to improve and the degree of capabilities. Therefore, capability levels (0-5) are used to express the capabilities and to measure the improvement within each selected process area. The basic concepts in CMMI for assessing the capability are goals and practices. To reach a capability level some goals must be achieved by conforming to some expected practices. CMMI differentiates between Specific Goals and Practices (SP), which are specific to a certain Process Area, and Generic Goals and Practices (GP), which are applicable to all Process Areas. 2 Selecting the relevant process areas The criteria for selecting the process areas are: 1. The process areas shall address a reasonable maturity level. 2. The supplier shall be responsible for performing the process area. 3. The process of the supplier has an interface to a Vodafone process. Add 1.) In the regarded context it is enough that the supplier achieves a maturity level of 3. Thus all process areas which correspond to level 4 and 5 in the staged representation were omitted (Organizational Process Performance, Organizational Innovation and Deployment, Quantitative Project Management, Causal Analysis and Resolution). These may come into focus after all essential suppliers have reached capability level 3 in all relevant process areas. Add 2.) Based on work split it is necessary to distinguish for which process areas Vodafone is more responsible for and for which the supplier is mainly responsible. In the investigated organization the supplier contributes only minor parts to the process areas Requirement Development, Product Integration and Validation. The process areas Supplier Agreement Management and Integrated Project Management are mainly under control of Vodafone. Add 3.) Project Planning and Project Monitoring and Control are strongly connected to the Vodafone planning and control process. The Risk Management of the supplier has to be integrated into the overall Risk Management of the project at Vodafone. The requirements of Vodafone are handled by the Requirements Management of the supplier. The Technical Solution has to follow Vodafone development guidelines and has to be accepted by Vodafone. By Verification the supplier has to fulfill some predefined quality gates. Finally Configuration Management is strongly related with the Configuration Management of Operations. Each of these areas will be checked by a detailed questionnaire. The remaining process areas are checked by Generic Practices and not by individual questionnaires. The main reason for this is that Vodafone is mainly interested in the results of these processes and not on the processes themselves. For example regard the process area Organizational Training. Vodafone is mainly interested that the staff is well trained for the job they do, e.g. designing the architecture of a web application. How to arrange this is left to the supplier. Therefore Organizational Training is checked within other questionnaires through the generic practice Train People (GP 2.5). The same holds for the process areas Organizational Process Focus (GP 2.1 and GP 3.2), Organizational Process Definition (GP 3.1), Process and Product Quality Assurance (GP 2.9), Measurement and Analysis (GP 2.8), and Decision Analysis and Resolution (by SP).